Axiomatic approach to feature subset selection basedon

In this paper an axiomatic characterisation of feature subset selection is presented. Two axioms are presented: suuciency axiom | preservation of learning information, and necessity axiom | minimising encoding length. The suuciency axiom concerns the existing dataset and is derived based on the following understanding: any selected feature subset should be able to describe the training dataset without losing information, i.e., it is consistent with the training dataset. The necessity axiom concerns predictability and is derived from Occam's razor, which states that the simplest among diierent alternatives is preferred for prediction. The two axioms are then restated in terms of relevance in a concise form: maximising both the r(X; Y) and r(Y ; X) relevance. Based on the relevance characterisation, a heuristic selection algorithm is presented and experimented with. The results support the axioms.